Setting a personal status using augmented reality

ABSTRACT

A computer system, method, and computer readable product are provided for setting a personal status using augmented reality. In various embodiments, an augmented-reality computing device captures an image of a physical scene, which includes a person. The computing device then identifies the person, and accesses a personal status for that person. The computing device generates and displays an augmented reality image that displays the personal status in proximity to the person in the scene.

BACKGROUND

The present disclosure relates to augmented reality (AR), and morespecifically, it relates to setting a personal status using augmentedreality.

Augmented reality may combine, or augment, a view of the physical worldwith computer-generated elements. For instance, a video of the physicalworld captured by a smart phone's camera may be overlaid with acomputer-generated image when it is displayed on the smart phone'sscreen. This computer-generated image may be a still image or video.Additionally, augmented reality may be produced via other outputs, suchas via a sound outputted by the smart phone. There are also glasses-typeaugmented reality devices that, to a user wearing such a device, overlayan image on what he or she sees in the physical world through thoseglasses.

In addition to a video of the physical world, other aspects of thephysical world may be captured and used to augment reality. Theseaspects include accelerometer data of a device, a direction that thedevice is pointed to based on compass data, and a physical location ofthe device based on GPS (Global Positioning System) data.

SUMMARY

In various embodiments, a computing device that implements an embodimentof the present disclosure, captures an image of a physical scene, thephysical scene including a first person. The computing device thendetermines an identity of the first person based on analyzing the image.

After determining the identity of the first person, the computing devicedetermines that there is a personal status for the first person based onthe determined identity of the first person. Then, the computing devicegenerates an augmented-reality image based on the personal status forthe first person such that the personal status is displayed in alocation proximal to the first person.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings are of illustrative embodiments. They do not illustrate allembodiments. Other embodiments may be used in addition or instead.Details that may be apparent or unnecessary may be omitted to save spaceor for more effective illustration. Some embodiments may be practicedwith additional components or steps and/or without all of the componentsor steps that are illustrated. When the same numeral appears indifferent drawings, it refers to the same or like components or steps.

FIG. 1 illustrates examples of hardware used according to embodiments ofthe present disclosure.

FIG. 2 illustrates an example of an augmented-reality image thatdisplays a personal status that may be generated according toembodiments of the present disclosure.

FIG. 3 illustrates an example user interface for setting a personalstatus according to embodiments of the present disclosure.

FIG. 4 illustrates an example system architecture incorporating acentral server according to embodiments of the present disclosure.

FIG. 5 illustrates an example system architecture incorporatingpeer-to-peer communication according to embodiments of the presentdisclosure.

FIG. 6 illustrates an example process for generating an augmentedreality image that includes a personal status according to embodimentsof the present disclosure.

FIG. 7 illustrates an example process for updating an augmented realityimage that includes a personal status according to embodiments of thepresent disclosure.

FIG. 8 illustrates an example process for generating an augmentedreality image that includes a personal status that has a permissionassociated with it according to elements of the present disclosure.

DETAILED DESCRIPTION

Detailed embodiments of the claimed structures and methods are disclosedherein. However, it may be understood that the disclosed embodiments aremerely illustrative of the claimed structures and methods that may beembodied in various forms. The present disclosure may, however, beembodied in many different forms and should not be construed as limitedto the example embodiments set forth herein. Rather, these exampleembodiments are provided so that this disclosure will be thorough andcomplete and will fully convey the scope of the present disclosure tothose skilled in the art. In the description, details of well-knownfeatures and techniques may be omitted to avoid unnecessarily obscuringthe presented embodiments.

Handling questions and comments in a group setting, like a meeting orpresentation, can be challenging for both the person who currently hasthe floor and the person who wishes to interject. For example, theperson who has the floor may lose his or her place in what he or she issaying. Some ways that someone may interject include, raising a hand,typing a question into a web meeting room, and waiting until the end ofthe presentation to say something—and, if embarrassed, the person whowishes to interject may not say anything at all.

A solution to handling questions and comments in a group setting, then,may be effectuated using augmented reality techniques. For example,Person A may wear an augmented reality device, such as aneyeglasses-type augmented reality device. Person B may set a personalstatus on a computer, along with a viewing access for the status (e.g.,who may view the status). Person A's augmented reality device mayrecognize Person B, then retrieve Person B's status, and display it nearPerson B's status. Person B may change and/or clear his or her status asdesired. These techniques may be applied to more than a single person'sstatus. For example, in a lecture hall with many students, Person A mayuse a plurality of personal statuses from the students to gauge classparticipation.

FIG. 1 illustrates an example of hardware used according to anembodiment of the present disclosure. As shown in FIG. 1, a blockdiagram illustrates examples of computer hardware that may be usedaccording to embodiments of the present disclosure for setting apersonal status using augmented reality. In some embodiments, thecomputer hardware of FIG. 1 may be embodied in eyewear warn by a user onhis or her face.

CPU (central processing unit) 104, RAM (random access memory) 106,persistent storage 108, input device 110, display 112, communicationsinterface 114, GPU (graphics processing unit) 116, and camera 118 areconnected to a system bus 102. It may be appreciated that system bus 102is presented logically and simplified, and that two or more of thesecomponents may be communicatively coupled by one or more separate buses.It also may be appreciated that the depictions of CPU 104 and GPU 116are simplified to emphasize the components that are depicted—for examplethey omit hardware that controls the flow of computer-executableinstructions within them.

In FIG. 1, persistent storage 108, in an embodiment of the presentdisclosure, has capabilities that include storing a program that canexecute the processes described herein. Persistent storage 108, in anembodiment of the present disclosure, can store an executing applicationthat effectuates setting a personal status using augmented reality, aswell as viewing another user's personal status using augmented reality.

Additionally, in FIG. 1, an input device 110, such as a keyboard and amouse may be used to provide input to the computer hardware of FIG. 1.For example, this input may be made by a user to identify that user'spersonal status. Where the hardware takes the form of eyeglasses, theinput device may take the form of a microphone used to capture voiceinput or hardware buttons on the eyeglasses, for example.

In one embodiment, the communications interface 114 of FIG. 1 isconnected to a communications network using a Wi-Fi or LTE (long-termevolution) network communications protocol. Communications interface 114may also comprise a network interface card (NIC) that is connected to acommunications network via an Ethernet cable. In the present disclosure,communications interface 114 may be used to transmit a user status toone or more other computers, which may include a central computer thatstores statuses and conveys them to hardware used by other users.Communications device 114 may also be used to receive information aboutother users' statuses. These external computers may be accessible via acommunications network.

In the present disclosure, communications interface 114 may receiveprocessing requests in accordance with a communication protocol, forexample TCP/IP (Transmission Control Protocol/Internet Protocol), fromanother computer (not shown), and processing results are sent to a thirdcomputer (not shown). As depicted, communications interface 114 maycomprise hardware for transmitting and receiving network data, and/orprocessor-executable instructions for doing the same.

FIG. 2 illustrates an example of an augmented-reality image thatdisplays a personal status that may be generated according to someembodiments of the present disclosure. The augmented-reality image ofFIG. 2 may be generated by performing the process of FIG. 6.

As depicted, there are three people in augmented reality image200—person 202 a, person 202 b, and person 202 c. Then, each person hasan associated personal status that is being displayed—person 202 a haspersonal status 204 a, person 202 b has personal status 204 b, andperson 202 c has personal status 204 c. The manner in which these threepersonal statuses are being rendered and displayed illustrates examplesof how personal statuses may be conveyed, though it may be appreciatedthat there may be more ways in which personal statuses may be conveyed.

Here, status 204 a is a question and it is displayed in a first textstyle (such as a first color). Then, status 204 b is a comment, and itis displayed in a different text style (such as a second color) relativeto status 204 a, as well as stylized in the form of a talk bubble ratherthan bare text. It may be useful to display comments and questions in adifferent manner, so that a user viewing augmented reality image 200 caneasily scan to distinguish between one and the other.

Note also that status 204 a and status 204 b are displayed in differentpositions relative to their corresponding user (user 202 a and user 202b, respectively). While status 204 a is displayed over the head of user202 a, status 204 b is displayed up and to the right of user 204b—again, in a comment bubble as well. These two positioning of statusrelative to a user are by way of example and there are other ways thatstatuses may be displayed. For instance, a status may be displayedacross a person's torso.

For example, status 204 c for user 202 c differs from status 204 a andstatus 204 b because status 204 a is not text. Here, status 204 c is anoval that surrounds user 202 c, to convey that user 202 c has a questionor comment, and would like to be called upon by a presenter. There areother ways that statuses may be conveyed without text, such as by usingemoji or images.

Augmented reality image 200 may comprise these personal statusesoverlaid upon on an image of a physical scene that is captured by acomputer and displayed on a computer display. In other embodiments, suchas where the computing device that generates the augmented reality imagetakes the form of eyeglasses, it may be that the user of the augmentedreality device is himself or herself looking at this physical scene thatwas captured, so the personal statuses are rendered and displayed in theuser's field of view while looking at the physical scene directly.

In some embodiments, a personal status for a person may be displayeddifferently to a second person depending on whether the second personhas already viewed this personal status. For example, the first time thesecond person views the personal status, it may be displayed as above,and any subsequent time the second person views that personal status, itmay be displayed with an icon indicating that it has been previouslyviewed. Or vice versa, where the icon is shown the first time the secondperson views the personal status and the icon is then not shownsubsequently. In other embodiments, the opacity of the personal statusmay be changed between the initial time a particular person views it andsubsequent times. It may be appreciated that there may be othertechniques used to distinguish between an initial time someone views aparticular status, and subsequent times.

FIG. 3 illustrates an example user interface for setting a personalstatus according to embodiments of the present disclosure. In someembodiments, the user interface of FIG. 3 may be implemented in a webpage or an app that is implemented upon the computer hardware of FIG. 1.This user interface comprises input field 302, where a user may inputhis or her personal status.

Then, a user may also specify a permission associated with viewing thepersonal status, which may limit who is (or which user accounts are)able to access and view the personal status. A user of the userinterface may select that the permission is public 304 a (that anyonemay access and view the status), Presenter 304 b (that only the personor people who are acting as a presenter in the current setting mayaccess and view the status), or Specific Person 304 c (that only one ormore specific people may access and view the status)—with selectionfield 304 d being used to identify the specific person or persons whohave this permission.

Once a user has identified a personal status as well as an associatedpermission in the user interface, the user may select a share button306, which shares this personal status with the specified people (suchas by uploading an indication of the personal status and associatedpermissions to central server 402 of FIG. 4).

FIG. 4 illustrates an example system architecture including a centralserver according to some embodiments of the present disclosure. Thisexample architecture comprises central server 402, augmented realitydevice and user 404, augmented reality device and user 406, augmentedreality device and user 408, and communications network 410. Each of thecentral server 402 and the three depicted augmented reality devices maybe implemented on an instance of the computer hardware of FIG. 1. In oneembodiment, the communications network may be the INTERNET.

In this example system architecture, when a user sets a personal status(such as via the user interface of FIG. 3), that personal status may besent across communications network 410 to central server 402. Centralserver 402 may then process the personal status, and then send it to oneor more of the augmented reality devices (based on a permissionassociated with the personal status), so that the augmented realitydevices may create an augmented reality image that includes the personalstatus. This augmented reality image may be augmented reality image 200of FIG. 2.

While each user is depicted here as using an augmented reality device,it may be appreciated that there are embodiments where a non-augmentedreality device, such as a laptop computer, is used by a user to set apersonal status. This central server architecture of FIG. 4 may becontrasted with a peer-to-peer architecture as depicted in FIG. 5.

FIG. 5 illustrates an example system architecture incorporatingpeer-to-peer communication according to embodiments of the presentdisclosure. Whereas in FIG. 4 there was central server 402 to receiveand then propagate personal statuses to augmented reality devices, asdepicted, no central server appears in FIG. 5. In contrast the threedepicted augmented reality devices (here depicted with a correspondinguser)—augmented reality device 504, augmented reality device 506, andaugmented reality device 508—may directly communicate acrosscommunications network 510, which may be similar to communicationsnetwork 410 of FIG. 4.

Each of these augmented reality devices 506-510 may register itsexistence with the other augmented reality devices according to apeer-to-peer discovery protocol. Then, when a user enters a personalstatus, that personal status may be propagated to other augmentedreality devices without the use of a central server. It may beappreciated that there may be other system architectures used forsetting a personal status using augmented reality, such as those thatmay combine a central server (such as to store personal statuses forlater, as described further on in the disclosure) as well as a p2pprotocol for conveying personal statuses between augmented realitydevices.

FIG. 6 illustrates an example process for generating an augmentedreality image that includes a personal status according to embodimentsof the present disclosure. In some embodiments, the process of FIG. 6may be implemented on the computer hardware of FIG. 1 to effectuatesetting a personal status using augmented reality. In some embodiments,the process of FIG. 6 may be implemented to effectuate acomputer-implemented method for viewing a personal status usingaugmented reality.

It may be appreciated that the process of FIG. 6 is an example process,and that there may be embodiments that implement more or feweroperations than are disclosed. It may also be appreciated that there maybe embodiments that implement the operations of FIG. 6 in a differentorder than they are depicted in FIG. 6.

It may further be appreciated that the operations of FIG. 6 may beimplemented in conjunction with the operations of other figures. Forexample, the operations of FIG. 6 may be implemented in conjunction withthe operations of FIG. 7, which may provide for updating an augmentedreality image that has personal statuses as the user of the augmentedreality device, or a person captured by the augmented reality device,moves over time.

By way of another example, the operations of FIG. 6 may be implementedin conjunction with the operations of FIG. 8, which may provide forgenerating an augmented-reality image with personal statuses, based onwhich personal statuses a given user account has permissions to access.

The process of FIG. 6 begins with operation 602 and moves to operation604. Operation 604 depicts capturing an image. Where the process of FIG.6 is implemented by the computer hardware of FIG. 1, camera 118 maycapture this image, and this image may either be a discrete image, orpart of a video that is being captured by the camera. In someembodiments, operation 604 may comprise capturing an image of a physicalscene, the physical scene including a first person. After operation 604,the process of FIG. 6 moves to operation 606.

Operation 606 depicts identifying a person in the captured image. Insome embodiments, possible faces of people in the captured image may beidentified, and these possible faces may be compared against a storeddatabase of known faces to identify the faces in the image. This form offacial recognition may be performed according to a variety oftechniques, such as by identifying and comparing features of a face(like a relative position, size, and/or shape of a nose, eyes, mouth, orears on a face), or three-dimensional face recognition (where camera 118is a three-dimensional camera).

In some embodiments, a person may be identified in the captured imageusing techniques other than facial recognition. For example, a physicallocation of a person may be determined based on physically locating acomputing device that the person has with him or her, such as a cellulartelephone. The physical location of a person's computing device (andtherefore, the person himself or herself where there is a knownassociation between the computing device and the person) may bedetermined using Wi-Fi triangulation, where Wi-Fi access points maydetermine a distance between the access point and the computing device,and/or an angle between the access point and the computing device. Thisangle and distance information from one or more Wi-Fi access points(along with information about a physical location of the access points)may generally be combined to determine a physical location for thecomputing device.

A similar approach to determining the position of a computing device(and therefore, the owner of the device) may be performed usingBLUETOOTH LOW ENERGY (BLE) beacons. Interaction between the computingdevice and a BLUETOOTH beacon using a BLUETOOTH protocol may be used todetermine the computing device's physical position relative to thatbeacon. Then, that relative positioning information may be combined withother sources of information about the computing device's physicallocation (such as the device's relative position to one or more otherBLUETOOTH beacons) to establish a physical location for the computingdevice. While the embodiments described herein primarily involve use offacial recognition to identify a person, it may be appreciated thatthese techniques, as well as others, may be used as well to identify aperson.

In some embodiments, operation 606 may comprise determining an identityof the first person based on performing facial recognition on the image.In some embodiments, operation 606 may comprise determining an identityof a first person based on performing facial recognition on a capturedimage of a physical scene, the captured image comprising an image of thefirst person. After operation 606, the process of FIG. 6 moves tooperation 608.

Operation 608 depicts determining whether there is a personal status forthe identified person. Where operation 608 is implemented by anaugmented reality device, the augmented reality device may determinewhether there is such a personal status in a variety of ways, such asdepending on whether there is a central server or peer-to-peer (p2p)architecture, and whether the mechanism by which the augmented realitydevice determines that there is a new status is a push or a pullmechanism.

For example, using a central server architecture (such the one depictedin FIG. 4) with a pull mechanism, when the identified person creates orupdates a personal status, that personal status is sent to the centralserver, and then the augmented reality device queries the central serverfor a personal status for the identified user upon identifying the user.By way of another example, using a p2p architecture with a pushmechanism, when the identified person creates or updates the personalstatus, a computing device of that identified person then pushes theidentified status to the augmented reality device, and the augmentedreality device may store that personal status and then access it here inoperation 608.

In some embodiments, operation 608 may comprise, in response todetermining the identity of the first person, determining whether thereis a personal status for the first person based on the determinedidentity of the first person. If it is determined in operation 608 thatthere is a personal status for the identified person, then the processof FIG. 6 moves to operation 610. Instead, if it is determined inoperation 608 that there is not a personal status for the identifiedperson, then the process of FIG. 6 moves to operation 612.

Operation 610 is reached from operation 608 where, in operation 608, itis determined that there is a personal status for the identified person.Operation 610 depicts adding the personal status to an augmented realityimage. In some embodiments, adding the personal status to an augmentedreality image may comprise storing an indication of the personal status,along with other personal statuses that are to be added to the image(through looping through operations 606-612 to examine each person inthe captured image). Then, once all personal statuses to be included inan augmented reality image have been identified, the augmented realityimage that includes those personal statuses may be generated inoperation 614.

In addition to generating an augmented reality image based on a personalstatus for one or more people in a captured image, these personalstatuses may be saved for later, for example so that a lecturer mayaccess the questions and comments from the students in his or her class.Saving personal statuses for later may be done for an individualperson's statuses, such as by saving an indication of the personalstatus in a computer memory, saving an indication of a second personalstatus for the first person in a computer memory, and displaying theindication of the personal status and the indication of the secondpersonal status on a computer display, independent of a physicallocation of the first person.

Saving personal statuses for later may be done for multiple peopletogether (e.g., the students in a given class), such as by saving anindication of the personal status in a computer memory, saving anindication of a second personal status for a second person in a computermemory, displaying the indication of the personal status and theindication of the second personal status on a computer display,independent of a physical location of the first person and a physicallocation of the second person. After operation 610, the process of FIG.6 moves to operation 612.

Operation 612 is reached from operation 610, or from operation 608 upondetermining that there is not a personal status for the identifiedperson. Operation 612 depicts determining whether there is anotherperson in the image. Determining whether there is another person in theimage may be performed using facial recognition techniques, such asthose described with respect to operation 606, to identify landmarks(such as a nose, mouth, or eyes) in the image that are indicative of aface.

While the process of FIG. 6 depicts identifying the people in thecaptured image as occurring one at a time, with a personal status thenbeing identified for that person, it may be appreciated that there arealternate embodiments. For example, in some embodiments, each person inthe captured image may first be identified before a personal status foreach of those people is identified.

Upon determining that there is another person in the image, the processof FIG. 6 returns to operation 606. Through this loop from operations606-612, each person in the image may be identified, and it may bedetermined if each person has a personal status set. However, upondetermining that there is not another person in the image, then theprocess of FIG. 6 moves to operation 614.

Operation 614 depicts generating an augmented reality image. In someembodiments, the augmented reality image may be generated based oncombining the captured image overlaid with each personal status for thepeople in the image. In other embodiments, the augmented reality devicemay be able to overlay images onto what its user is actually looking at(such as with an eyeglasses-type augmented reality device) rather thanonto a captured image. In these embodiments, generating the augmentedreality image may include generating an image that comprises eachpersonal status, where those personal statuses are rendered in alocation within the user's field of view that corresponds to the personwhose personal status it is.

In some embodiments, operation 614 comprises generating anaugmented-reality image based on the personal status for the firstperson, such that the personal status is displayed in a locationproximal to the first person. In some embodiments, the location proximalto the first person is a location above the first person's head.Operation 614 may comprise generating the augmented-reality image basedon the personal status for the first person with the personal statuscolored with a first color in response to determining that the firststatus is indicative of a question, and generating the augmented-realityimage based on the personal status for the first person with thepersonal status colored with a second color in response to determiningthat the first status is indicative of a comment.

The personal status of one or more people may be updated (for exampleeither to a new status, or cleared) after the augmented reality image isgenerated in operation 614. In such embodiments, updating a personalstatus may comprise, after generating the augmented-reality image,determining that the personal status of the first person has beenupdated, and generating a second augmented-reality image based on theupdated personal status for the first person such that the updatedpersonal status is displayed in a location proximal to the first person.

Where a personal status is updating by way of clearing the personalstatus, so that there is no longer an active personal status associatedwith a given person, clearing the personal status may be effectuated by,after generating the augmented-reality image, determining that thepersonal status of the first person has been updated to indicate nopersonal status, and generating a second augmented-reality image basedon the updated personal status for the first person such that the secondaugmented reality image shows no personal status for the first person.After operation 614, the process of FIG. 6 moves to operation 616, wherethe process of FIG. 6 ends.

FIG. 7 illustrates an example process for updating an augmented realityimage that includes a personal status according to embodiments of thepresent disclosure. In some embodiments, the operations of FIG. 7 may beimplemented on the computer hardware of FIG. 1 to effectuate setting apersonal status using augmented reality. It may be appreciated that theoperations of FIG. 7 are example operations, and that there may beembodiments that implement more or fewer operations than are disclosed.It may also be appreciated that there may be embodiments that implementthe operations of FIG. 7 in a different order than they are depicted inFIG. 7.

The process of FIG. 7 begins with operation 702 and then moves tooperation 704. Operation 704 depicts capturing a second image. In someembodiments, operation 704 may be implemented in a manner similar tothat of capturing an image, as performed in operation 604 of FIG. 4.This second image may generally be considered to be captured at a timesubsequent to a time when a previous image was captured. Where theimages are part of a video, this second image may be a frame of thevideo that is subsequent to a prior frame of the video.

In some embodiments, operation 704 may comprise capturing a secondimage, the first person being in a different location in the secondimage than in the first image. After operation 704, the process of FIG.7 moves to operation 706.

Operation 706 depicts selecting a person depicted in the second image.Selecting a person depicted in the second image may be performed in amanner similar to that of operation 606 of FIG. 6 (i.e., identifying aperson in a captured image), and operation 612 of FIG. 6 (determiningwhether there is another person in the captured image). After operation706, the process of FIG. 7 moves to operation 708.

Operation 708 depicts determining whether the selected person has movedin the second image relative to a previous image. There may be a fewgeneral ways that a person may move position within an image between twoimages. In various scenarios, the person may have moved, the augmentedreality device that captures the person may have moved, both the personand the augmented reality device may have moved, and another person orobject may have moved between the person and the augmented realitydevice so that the person is now obfuscated in the image. Determiningwhether the selected person has moved between the two images may then beperformed by a comparison of the two images, where the person isidentified in each image and the position within each image of theperson is compared.

Upon determining in operation 708 that the selected person has moved inthe second image relative to the previous image, then the process ofFIG. 7 moves to operation 710. Instead, upon determining in operation708 that the selected person has not moved in the second image relativeto the previous image, then the process of FIG. 7 moves to operation714.

Operation 710 is reached from operation 708 upon determining inoperation 708 that the selected person has moved in the second imagerelative to the previous image. Operation 710 depicts determiningwhether there is a personal status for the selected person. In someembodiments, operation 710 may be implemented in a similar manner asimplementing operation 608 of FIG. 6.

If it is determined in operation 710 that there is a personal status forthe selected person, then the process of FIG. 7 moves to operation 712.Instead, if it is determined in operation 710 that there is not apersonal status for the selected person, then the process of FIG. 7moves to operation 714.

Operation 712 is reached from operation 710, upon determining that thereis a personal status for the person. Operation 712 depicts moving thepersonal status for the selected person in a subsequent augmentedreality image. So, as personal statuses are displayed using augmentedreality, those personal statuses may be rendered in augmented realityimages in a fixed location relative to their corresponding person—forinstance, some distance above the person's head. As the person moveswithin the captured image, the position at which the personal status isrendered may be correspondingly moved when rendering the augmentedreality image, so that this relation between the position of the personand the position of the personal status remains constant.

In some embodiments, there may be other considerations that affect thepositioning of a personal status in an augmented reality image. Forexample, if two people in the image with personal statuses are standingin close proximity, it may be that rendering their statuses according tothe default status-positioning behavior would result in one statusobfuscating another status. In such embodiments, it may be that suchobfuscation is detected, and where it would result from the defaultbehavior, at least one of the statuses is rendered in a differentlocation than the default so that no obfuscation occurs. After operation712, the process of FIG. 7 moves to operation 714.

Operation 714 is reached from operation 712, from operation 708 upondetermining that the selected person did not move in the second imagerelative to a prior image, and from operation 710 upon determining thatthere is not a personal status for the selected person. Operation 714depicts determining whether there is another person depicted in theimage. In some embodiments, operation 714 may be implemented in asimilar manner as operation 612 of FIG. 6, which also depictsdetermining whether there is another person depicted in an image.

If it is determined in operation 714 that there is another person in theimage, the process of FIG. 7 returns to operation 708. Instead, if it isdetermined in operation 714 that there is not another person in theimage, then the process of FIG. 7 moves to operation 716.

Operation 716 is reached from operation 714, upon determining that thereis not another person in the image. Operation 716 depicts generating anupdated augmented reality image. Generating this updated reality imagemay comprise generating an augmented reality image based on the movedstatuses that are determined from operation 712, as well as any statusesthat have not moved (because the corresponding person has not movedwithin the captured image relative to a prior captured image).

In some embodiments, operation 716 may comprise generating a secondaugmented-reality image based on the personal status for the firstperson such that the personal status is displayed in a location proximalto the first person being in the different location. After operation716, the process of FIG. 7 moves to operation 718, where the process ofFIG. 7 ends.

FIG. 8 illustrates an example process for generating an augmentedreality image that includes a personal status that has a permissionassociated with it according to elements of the present disclosure. Insome embodiments, the operations of FIG. 8 may be implemented on thecomputer hardware of FIG. 1 to effectuate setting a personal statususing augmented reality. For example, the operations of FIG. 8 may beimplemented on central server 402 of FIG. 4 or by one of augmentedreality devices 504-508 of FIG. 5 to determine whether to send aparticular personal status to another augmented reality device, based onits permissions.

It may be appreciated that the operations of FIG. 8 are exampleoperations, and that there may be embodiments that implement more orfewer operations than are disclosed. It may also be appreciated thatthere may be embodiments that implement the operations of FIG. 8 in adifferent order than they are depicted in FIG. 8.

The process of FIG. 8 begins with operation 802, and moves to operation804. Operation 804 depicts determining whether there is an unexaminedaugmented reality device. In some embodiments, a list or other datastructure includes an indication of each augmented reality device. Thislist may be maintained by a central server, like central server 402 ofFIG. 4, or by each augmented reality device where a p2p architecture isused as in FIG. 5. An augmented reality device may register itself tothe central server or other augmented reality devices when it connectsto a corresponding communications network, or otherwise performsoperations to receive and display personal statuses using augmentedreality. Once an augmented reality device registers itself, the centralserver or other augmented reality devices may store an indication ofthat registered augmented reality device in the list. Then, determiningwhether there is an unexamined augmented reality device may comprisetraversing the list, and determining whether the end of the list hasbeen reached (which may indicate that there is not an un-examinedaugmented reality device).

If it is determined in operation 804 that there is an unexaminedaugmented reality device, then the process of FIG. 8 moves to operation806. Instead, if it is determined in operation 804 that there is not anunexamined augmented reality device, then the process of FIG. 8 moves tooperation 820, where the process of FIG. 8 ends.

Operation 806 is reached from operation 804, upon determining that thereis an unexamined augmented reality device. Operation 806 depictsselecting an augmented reality device. Using the example of maintaininga list of augmented reality devices from operation 804, operation 806may comprise selecting the next unexamined augmented reality device inthis list. After operation 806, the process of FIG. 8 moves to operation808.

Operation 808 depicts determining if there is an unexamined personalstatus. Where a central server maintains personal statuses for multipleusers, these multiple personal statuses may be stored in a list, similarto the list of augmented reality devices described in operation 804, anddetermining whether there is an unexamined personal status may beperformed in a manner similar to determining whether there is anunexamined augmented reality device in operation 804.

Where there is a p2p architecture (such as the one depicted in FIG. 5),it may be that each augmented reality device maintains at most onecurrent status, and determining whether there is an unexamined personalstatus comprises determining whether this one current status both existsand has been examined.

If it is determined in operation 808 that there is an unexamined status,then the process of FIG. 8 moves to operation 810. Instead, if it isdetermined in operation 808 that there is not an unexamined status, thenthe process of FIG. 8 returns to operation 804.

Operation 810 is reached from operation 808, upon determining that thereis an unexamined status. Operation 810 depicts selecting an unexaminedstatus. In some embodiments, operation 810 may be implemented in asimilar manner as operation 806, which depicts selecting an unexaminedaugmented reality device. After operation 810, the process of FIG. 7moves to operation 812.

Operation 812 depicts determining the status permission. In embodiments,each personal status may be stored in conjunction with an indication ofa permission required to view this status. Then, when the personalstatus is accessed, the corresponding status permission may be accessedand analyzed. This permission may be, for example, that anyone mayaccess the personal status, that a specific person may access thepersonal status, or that someone who is a presenter may access thepersonal status (such as a lecturer in a classroom, or a speaker at aconference; in this case, the permission is associated with a person'srole as a presenter rather than the person's identity as a specificperson). In addition, the permission may comprise combinations of thesepermissions—for example, the permission associated with a particularstatus may be for two specific people as well as whoever the presenteris.

In some embodiments, operation 812 may lead to generating theaugmented-reality image based on the personal status for the firstperson in response to determining that a user account associated with anaugmented-reality device used to generate the augmented-reality imagehas permission to access the personal status for the first person. Insome embodiments, upon determining that anyone has permission to accessthe personal status, operation 812 may lead to generating theaugmented-reality image based on the personal status for the firstperson in response to determining that the personal status for the firstperson has a permission set to allow anyone to access the personalstatus.

Upon determining in operation 812 that the status permission is for aSpecific Person, then the process of FIG. 8 moves to operation 814.Instead, upon determining in operation 812 that the status permission isfor a Presenter, then the process of FIG. 8 moves to operation 816.Instead, upon determining in operation 812 that the status permission isfor anyone, then the process of FIG. 8 moves to operation 818.

Operation 814 is reached from operation 812, upon determining that thestatus permission is for a Specific Person. Operation 814 depictsdetermining if the selected augmented reality device is associated withthe specific person. It may be that a person logs in when using aparticular augmented reality device (such as with a user name andpassword), and then there is a corresponding user account that theperson logged into. This user account that is active on a givenaugmented reality device may be used to determine whether the augmentedreality device is associated with a particular specific person.

In some embodiments, upon determining in operation 814 that the selectedaugmented reality device is associated with the specific person,operation 814 may lead to generating the augmented-reality image basedon the personal status for the first person in response to determiningthat the personal status for the first person has a permission set toallow a first user account to access the personal status, and the useraccount is associated with an augmented-reality device used to generatethe augmented-reality image.

Where in operation 814 it is determined that the selected augmentedreality device is associated with the specific person, then the processof FIG. 8 moves to operation 816. Instead, upon determining in operation814 that the selected augmented reality device is not associated withthe specific person, then the process of FIG. 8 returns to operation808.

Operation 816 is reached from operation 812, upon determining that thestatus permission is for a Presenter. Operation 816 depicts determiningwhether the augmented reality device is associated with a Presenterstatus. Continuing with the user account example of operation 814, acomputing device that implements the process of FIG. 8 may maintain anidentification of one or more user accounts that currently havePresenter status. Then, operation 816 may be implemented by determiningwhether the user account associated with the augmented reality device isone of these user accounts that currently does have Presenter status.

In some embodiments, upon determining in operation 816 that theaugmented reality device is associated with Presenter status, operation816 may lead to generating the augmented-reality image based on thepersonal status for the first person in response to determining that thepersonal status for the first person has a permission set to allow apresenter of an event the first person is attending to access thepersonal status, and a user account associated with an augmented-realitydevice used to generate the augmented-reality image has a status as apresenter of the event.

In some embodiments, the personal status has access permissions thatindicate that viewing the personal status is limited to user accountswith a presenter status. In such embodiments, a user account may haveand then lose presenter status (for example, if a person presents duringjust one hour of a day-long conference). In such embodiments, when auser account loses presenter status, this user account no longer hasaccess to personal statuses that require a Presenter status to access.This scenario of a user account losing presenter status may be addressedby, after generating the augmented-reality image, determining whether auser account associated with a computer that generated theaugmented-reality image no longer has presenter status, and in responseto determining whether the user account associated with the computerthat generated the augmented-reality image no longer has presenterstatus, halting presenting an augmented-reality image that includes thepersonal status.

Where in operation 816 it is determined that the augmented realitydevice is associated with a Presenter status, then the process of FIG. 8moves to operation 818. Instead, where in operation 816 it is determinedthat the augmented reality device is not associated with a Presenterstatus, then the process of FIG. 8 returns to operation 808.

Operation 818 is reached from operation 812 upon determining that thepermission status is for anyone, operation 814 upon determining that theaugmented reality device is associated with the specific person, andfrom operation 816 upon determining that the augmented reality device isassociated with a Presenter status. Operation 818 depicts sending thepersonal status to the selected augmented reality device.

In some embodiments, operation 818 may comprise central server 402 ofFIG. 4 or one of augmented reality devices 504-508 of FIG. 5 sending toanother augmented reality device an indication of the personal status,along with a person who corresponds to that personal status. Afteroperation 808, the process of FIG. 8 returns to operation 808.

Detailed embodiments of the claimed structures and methods are disclosedherein. However, it can be understood that the disclosed embodiments aremerely illustrative of the claimed structures and methods that may beembodied in various forms. The present disclosure may, however, beembodied in many different forms and should not be construed as limitedto the example embodiments set forth herein. Rather, these exampleembodiments are provided so that this disclosure will be thorough andcomplete and will fully convey the scope of the present disclosure tothose skilled in the art. In the description, details of well-knownfeatures and techniques may be omitted to avoid unnecessarily obscuringthe presented embodiments.

The present disclosure may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent disclosure.

The computer readable storage medium can be a tangible and/ornon-transitory device that may retain and store instructions for use byan instruction execution device. For example, the computer readablestorage medium may be, but is not limited to, an electronic storagedevice, a magnetic storage device, an optical storage device, anelectromagnetic storage device, a semiconductor storage device, or anysuitable combination of the foregoing. A non-exhaustive list of morespecific examples of the computer readable storage medium includes thefollowing: a portable computer diskette, a hard disk, a random accessmemory (RAM), a read-only memory (ROM), an erasable programmableread-only memory (EPROM or flash memory), a static random access memory(SRAM), a portable compact disc read-only memory (CD-ROM), a digitalversatile disc (DVD, alternatively known as a digital video disc), amemory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network(LAN), a wide area network (WAN), and/or a wireless network. The networkmay comprise copper transmission cables, optical transmission fibers,wireless transmission, routers, firewalls, switches, gateway computersand/or edge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present disclosure may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as a Smalltalk or C++ programminglanguage or the like, and conventional procedural programming languages,such as a C programming language or similar programming languages. Thecomputer readable program instructions may execute entirely on theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an (ISP) InternetService Provider). In some embodiments, electronic circuitry including,for example, programmable logic circuitry, field-programmable gatearrays (FPGA), or programmable logic arrays (PLA), may execute thecomputer readable program instructions by utilizing state information ofthe computer readable program instructions to personalize the electroniccircuitry, in order to perform aspects of the present disclosure.

Aspects of the present disclosure are described herein with reference toprocess or flowchart illustrations, and/or block diagrams of methods,apparatus (systems), and computer program products according toembodiments of the present disclosure, and these illustrations maycomprise one or more operations. It will be understood that each blockof the flowchart illustrations and/or block diagrams, and combinationsof blocks in the flowchart illustrations and/or block diagrams, can beimplemented by computer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present disclosure. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

What is claimed is:
 1. A computer-implemented method for viewing apersonal status using augmented reality, the method comprising:capturing an image of a physical scene, the physical scene including afirst person; determining an identity of the first person in the image;in response to determining the identity of the first person, determiningwhether there is a personal status for the first person based on thedetermined identity of the first person; generating an augmented-realityimage based on the personal status for the first person such that thepersonal status is displayed in a location proximal to the first person;after generating the augmented-reality image, determining whether a useraccount associated with a computer that generated the augmented-realityimage no longer has presenter status; and in response to determiningthat the user account associated with the computer that generated theaugmented-reality image no longer has presenter status, haltingpresenting the augmented-reality image, wherein the personal status hasaccess permissions that indicate that viewing the personal status islimited to user accounts with a presenter status.
 2. Thecomputer-implemented method of claim 1, further comprising: capturing asecond image, the first person being in a different location in thesecond image than in the first image; and generating a secondaugmented-reality image based on the personal status for the firstperson such that the personal status is displayed in a location proximalto the first person being in the different location.
 3. Thecomputer-implemented method of claim 1, wherein the location proximal tothe first person is above the first person's head.
 4. Thecomputer-implemented method of claim 1, further comprising: aftergenerating the augmented-reality image, determining whether the personalstatus of the first person has been updated; and generating a secondaugmented-reality image based on the updated personal status for thefirst person such that the updated personal status is displayed in alocation proximal to the first person.
 5. The computer-implementedmethod of claim 1, further comprising: generating the augmented-realityimage based on the personal status for the first person; and upondetermining that the first status is indicative of a question, coloringthe personal status with a first color.
 6. The computer-implementedmethod of claim 5, further comprising: generating the augmented-realityimage based on the personal status for the first person with thepersonal status colored with a second color, upon determining that thefirst status is indicative of a comment.
 7. (canceled)
 8. A computersystem for viewing a personal status using augmented reality,comprising: a processor, a computer-readable memory, a computer-readabletangible storage device, and program instructions stored on the storagedevice for execution by the processor via the memory, wherein executionof the program instructions by the computer system configures thecomputer system to: determine an identity of a first person depictedwithin a captured image of a physical scene; in response to determiningthe identity of the first person, determine whether there is a personalstatus for the first person based on the determined identity of thefirst person; generate an augmented-reality image based on the personalstatus for the first person such that the personal status is displayedin a location proximal to the first person; and generate theaugmented-reality image based on the personal status for the firstperson in response to determining that a user account associated with anaugmented-reality device used to generate the augmented-reality imagehas permission to access the personal status for the first person. 9.(canceled)
 10. The computer system of claim 8, wherein execution of theprogram instructions further configures the computer system to: generatethe augmented-reality image based on the personal status for the firstperson in response to determining that the personal status for the firstperson has a permission set to allow anyone to access the personalstatus.
 11. The computer system of claim 8, wherein execution of theprogram instructions further configures the computer system to: generatethe augmented-reality image based on the personal status for the firstperson in response to determining that (i) the personal status for thefirst person has a permission set to allow a presenter of an event thatthe first person is attending to access the personal status, and (ii) auser account associated with an augmented-reality device used togenerate the augmented-reality image has a status as a presenter of theevent.
 12. The computer system of claim 8, wherein execution of theprogram instructions further configures the computer system to: generatethe augmented-reality image based on the personal status for the firstperson in response to determining that (i) the personal status for thefirst person has a permission set to allow a first user account toaccess the personal status, and (ii) the user account is associated withan augmented-reality device used to generate the augmented-realityimage.
 13. The computer system of claim 8, wherein execution of theprogram instructions further configures the computer system to: capturea second image, the first person being in a different location in thesecond image than in the first image; and generate a secondaugmented-reality image based on the personal status for the firstperson such that the personal status is displayed in a location proximalto the first person being in the different location.
 14. The computersystem of claim 8, wherein execution of the program instructions furtherconfigures the computer system to: determine the identity of the firstperson depicted within the captured image of the physical scene based onperforming a facial recognition on the captured image, or based ondetermining a location of computing device associated with the firstperson. 15.-20. (canceled)
 21. A computer system for viewing a personalstatus using augmented reality, comprising: a processor, acomputer-readable memory, a computer-readable tangible storage device,and program instructions stored on the storage device for execution bythe processor via the memory, wherein execution of the programinstructions by the computer system configures the computer system to:determine an identity of a first person depicted within a captured imageof a physical scene; in response to determining the identity of thefirst person, determine whether there is a personal status for the firstperson based on the determined identity of the first person; generate anaugmented-reality image based on the personal status for the firstperson such that the personal status is displayed in a location proximalto the first person; and generate the augmented-reality image based onthe personal status for the first person in response to determining thatthe personal status for the first person has a permission set to allowanyone to access the personal status.